Recent advances in constrained reinforcement learning (RL) have endowed reinforcement learning with certain safety guarantees.

Neural Information Processing Systems http://nips.cc/

Given training data ofn-dimensional LPs, we learn ann k projection matrix withn>k.

The classical optimal transport (OT) problem seeks the map that moves mass from a source to a target measure while minimizing a prescribed cost function. The objective can be formalized in either Monge's [12] or Kantronich's formulation [10], a convex relaxation of the former that considers transport plans instead of deterministic maps. These foundational formulations have wide-ranging applications, including to economics [7] and machine learning [14]. In many practical scenarios, the source measure is known or readily in-ferrable from empirical data but the target measure is not explicitly specified. Instead, it is only constrained by practical requirements or expert knowledge. For example, when applying Monge's formulation to transportation problems, the placement of the mass in the target region may be constrained to lie entirely beyond a certain boundary or within a particular region, rather than by the specification of a precise location for each fraction of the total mass. Similarly, in economic applications, supply and demand may be subject to constraints such as maximal amounts available or minimal amounts required, rather than dictated through precise marginal distributions. 1

Speculative sampling reduces the latency of autoregressive decoding for target model LLMs without sacrificing inference quality, by using a cheap draft model to suggest a candidate token and a verification criterion to accept or resample this token. To improve acceptance and decoding efficiency, recent work has explored the multi-draft extension, where at each step $n$ draft tokens are generated, and the verification criterion is a distribution conditioned on these. When this criterion maximizes the probability of accepting some draft token, it is called the optimal transport (OT). However, finding the OT is difficult, as it is the solution of a linear program (OTLP) in over $V^n$ variables, with $V$ being the vocabulary size. Two recent theoretical works have reframed the OTLP in terms of importance sampling or subset selection. In this work, we prove that these formulations are equivalent to an exponentially large relaxed OTLP, so it remains infeasible to solve. Then, we reverse engineer subset selection to formulate the OTLP as a max-flow problem. With a novel application of polymatroid theory, we reduce the exponentially large OTLP to a convex optimization problem in at most $V$ variables. This allows us to devise an algorithm for optimal $n$-draft speculative sampling when the $n$ tokens are chosen i.i.d. from a single draft model, which can be tuned to arbitrary accuracy. Finally, we measure acceptance rates and algorithm runtimes for various $n$ and top-$k$ draft sampling settings. Our findings give the first multi-draft algorithm with 90% acceptance and under 100 ms of overhead per generated token with negligible deviation from the target model distribution.

Neural Information Processing Systems http://nips.cc/

The core principle of PlaCo is to provide a high-level interface for specifying robot control problems, while internally reformulating them into the QP formulation introduced in equation (1) expected by efficient numerical solvers. This section illustrates how common robotics problems naturally reduce to this form. First, Section III-A recalls the equivalence between least-squares objectives and the standard QP formulation. Section III-B extends this formulation to the case of multiple objectives. Section III-C discusses how to incorporate hard and soft constraints into the QP framework. Section III-D introduces integrated decision variables, which allow system dynamics to be embedded directly into the QP problem. Finally, Section III-E presents how QR factorization is used to reduce the dimensionality of the optimization problem. An usage example is provided in Appendix A to illustrate the problem specification process in PlaCo. A. From least-squares to standard QP formulation A least-squares minimization problem is formulated as min